A system and method is disclosed for detecting locally-generated or other unknown graphics that are superimposed on a video program displayed on a television receiver or other like device. Devices external to a television receiver, such as a cable or satellite set-top box, can generate logos, electronic program guides, or other images that are then graphically overlaid on the television signal emanating from the device to the television's display. Likewise, the new generation of smart TV's contain internal processors and graphics display means to display overlaid windows of information generated by smartphone-like applications on said internal processors. Said internal processing can also receive television programming from the Internet and display said programming in place of television program from traditional television broadcasters. Further, local broadcast or cable TV operators can overlay certain graphics or alerts superimposed on the television programming that they carry. In some instances, understanding what the graphics communicate is important such as programmer identification. In other cases, these graphics can result in impairing or disabling the video recognition capability of an automatic content recognition (ACR) system [such as with the invention of U.S. Pat. No. 8,595,781 of which this document is a continuation in part]. The system and method described herein provides a means to detect the presence of such on-screen graphics both to detect and identify graphical information and to also enable automated resolution of any interference among or between a plurality of video graphic sub-systems.
Television broadcast signals have long included static graphics such as logos, program identification, or other information formatted to display superimposed on underlying video programming. Often, these identifiers are small in size, semi-transparent and located in a corner of the display area. A common example is the logo of a broadcast network displayed in typically the lower right corner of a television picture. In recent decades, information about the television programing, such as the network, channel and the name of the program, has been embedded as metadata in the digital television signal and broadcast freely to television receivers. In other cases, television program information has been supplied from third-parties such as Tribune and Rovi as electronic program guides (EPG). This information can be formatted as graphical information for display on television receivers by cable and satellite set-top boxes.
Television sets, set-top boxes, and home computers are now converging into what are sometimes called “Smart TVs.” To take advantage of the full potential of this new technology, the computing means within or associated with the television set needs real-time “awareness” of the programing being displayed by it.
Various means to achieve that goal of such content awareness are known to those skilled in the art; including approaches to video fingerprinting that enable a computing means to match what is currently being displayed on the screen of the television to databases of possible candidates. A typical system that possesses this capability is described in U.S. Pat. No. 8,595,781.
However, a difficulty arises when additional, local components (such as a set-top box, game console or other connected devices) generate graphical user interfaces, text messages, or logos that display superimposed on the video programming. This overlaying of graphical elements, such as program information that the user has invoked from a cable or satellite set-top box, cause the video fingerprint recognition means to fail since the fingerprint matching data provided for the algorithm has no awareness of the additional locally-generated graphical screen elements and may return a “no match” result. It is the goal of this invention to enable the video matching sub-system to associate any such “no match” result with an “on-screen graphic interference” notification to enable the video matching system to respond appropriately. It is also a goal of the system operating on the processing means within a television system to signal information to a central content matching system to adjust to the presence of otherwise interfering graphical elements and only attempt to match video samples drawn from around the video display in areas not affected by locally generated graphic elements. It is still a further goal of the invention to identify overlaid graphical elements, such as television channel logos, using the processing means of the smart TV when advantageous to the system.